Artificial Intelligence Models for Scheduling Big Data Services on the Cloud

The widespread adoption of Internet of Things (IoT) applications in many critical sectors (e.g., healthcare, unmanned autonomous systems, etc.) and the huge volumes of data that are being generated from such applications have led to an unprecedented reliance on the cloud computing platform to store and process these data. Moreover, cloud providers tend to receive massive waves of demands on their storage and computing resources. To help providers deal with such demands without sacrificing performance, the concept of cloud automation had recently arisen to improve the performance and reduce the manual efforts related to the management of cloud computing workloads. However, several challenges have to be taken into consideration in order to guarantee an optimal performance for big data storage and analytics in cloud computing environments. In this context, we propose in this thesis a smart scheduling model as an automated big data task scheduling approach in cloud computing environments. Our scheduling model combines Deep Reinforcement Learning (DRL), Federated Learning (FL), and Transfer Learning (TL) to automatically predict the IoT devices to which each incoming big data task should be scheduled to as to improve the performance and reduce the execution cost. Furthermore, we solve the long execution time and data shortage problems by introducing a FL-based solution that also ensures privacy-preserving and reduces training and data complexity. The motivation of this thesis stems from four main observations/research gaps that we have drawn through our literature reviews and/or experiments, which are: (1) most of the existing cloud-based scheduling solutions consider the scheduling problem only from the tasks priority viewpoint, which leads to increase the amounts of wasted resources in case of malicious or compromised IoT devices; (2) the existing scheduling solutions in the domain of cloud and edge computing are still ineffective in making real-time decisions concerning the resource allocation and management in cloud systems; (3) it is quite difficult to schedule tasks or learning models from servers in areas that are far from the objects and IoT devices, which entails significant delay and response time for the process of transmitting data; and (4) none of the existing scheduling solutions has yet addressed the issue of dynamic task scheduling automation in complex and large-scale edge computing settings. In this thesis, we address the scheduling challenges related to the cloud and edge computing environment. To this end, we argue that trust should be an integral part of the decision-making process and therefore design a trust establishment mechanism between the edge server and IoT devices. The trust mechanism model aims to detect those IoT devices that over-utilize or under-utilize their resources. Thereafter, we design a smart scheduling algorithm to automate the process of scheduling large-scale workloads onto edge cloud computing resources while taking into account the trust scores, task waiting time, and energy levels of the IoT devices to make appropriate scheduling decisions. Finally, we apply our scheduling strategy in the healthcare domain to investigate its applicability in a real-world scenario (COVID-19)

A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks

Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. Unlike conventional neural networks or updated versions of Recurrent Neural Networks (RNNs) such as Long Short-Term Memory (LSTM), transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. As a result, transformer-based models have attracted substantial interest among researchers in the field of artificial intelligence. This can be attributed to their immense potential and remarkable achievements, not only in Natural Language Processing (NLP) tasks but also in a wide range of domains, including computer vision, audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published highlighting the transformer's contributions in specific fields, architectural differences, or performance evaluations, there is still a significant absence of a comprehensive survey paper encompassing its major applications across various domains. Therefore, we undertook the task of filling this gap by conducting an extensive survey of proposed transformer models from 2017 to 2022. Our survey encompasses the identification of the top five application domains for transformer-based models, namely: NLP, Computer Vision, Multi-Modality, Audio and Speech Processing, and Signal Processing. We analyze the impact of highly influential transformer-based models in these domains and subsequently classify them based on their respective tasks using a proposed taxonomy. Our aim is to shed light on the existing potential and future possibilities of transformers for enthusiastic researchers, thus contributing to the broader understanding of this groundbreaking technology